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PRIORITY CLAIM 

[0001] The present application claims priority to UK Patent Application 
No. 03 026 64.8, entitled "Microscope System and Method", which application 
was filed on February 5, 2003. 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0002] The present invention relates to the field of microscopy, and in 
particular to an aid for a microscope user. 

Description of the Related Art 

[0003] Microscopes are commonly used to view images of very small objects. 
A typical optical microscope has several objective lenses, of different magnifying 
power, to enable an object on a microscope slide to be viewed at different 
magnifications. 

[0004] A problem with viewing an object through a microscope is that only a 
portion of the object can be seen in the field of view at any one time. The higher 
the magnifying power of the objective lens, the smaller the area of the object 
visible in the field of view. Thus, at very high magnifications (e.g. use of a 20x (or 
above) objective lens), the field of view only contains a very small part of the 
complete object. Consequently, characteristic features of the object, such as its 
outline shape, or the distinctive shape of features within the object, are unlikely to 
be visible in the field of view, or, if visible are unlikely to be of a size sufficient to 
enable determination of the position of the visible area within the overall object. 
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[0005] When using a microscope for studying a biological specimen, for 
example, a user will typically look at the specimen on the microscope slide with 
the naked eye (e.g. by holding the slide up to the light) to get a sense of the 
outline shape of the specimen and the location of distinctive visible features 
5 within the outline. The user may then view the object through the microscope at 
low magnification to more clearly identify the main distinctive features of the 
object before viewing the object in detail at high magnification. 

[0006] When viewing the specimen at high magnification, the user estimates 
10 the position of the portion of the specimen in the field of view within the overall 
specimen based on his or her memory of the shape of the specimen. As a result, 
even the most experienced microscope users find it difficult to determine the 
exact position of a part of the specimen contained within a field of view in relation 
to the overall specimen, especially at high magnifications. 

15 

[0007] Recently, developments have been made in the field of "virtual 
microscopy" in which digital images of an object are captured from a microscope 
by a digital camera and assembled together by image processing software to 
form an overall image of the object which can be displayed on a display screen. 

20 An example of a virtual microscopy technique is described in EP-A-0 994 433 in 
the name of Fairfield Imaging Limited. In the described technique, adjacent 
images are captured from a microscope at high magnification/resolution and are 
assembled together to provide a composite image. A low 
magnification/resolution copy of the image is obtained by image processing 

25 which is displayed on a display screen and used as a "navigation map". A user 
can select, from the navigation map, a portion of the low magnification/resolution 
image for viewing at high resolution. Thus, the user of a virtual microscope has 
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the benefit of knowing the positional relationship of a displayed high 
magnification image relative to the overall object. 

[0008] Although virtual microscopy is increasingly used, it necessitates the 
5 provision of an expensive, high precision microscope that is dedicated to imaging 
microscope slides, along with associated equipment including a stage driver, 
high quality digital or video camera and associated workstation. Such additional 
equipment can be bulky, and can inhibit normal use of the microscope when not 
performing imaging for virtual microscopy. 

10 

[0009] It would be desirable to provide a system and method which can be 
used with a conventional microscope that provides some of the aforementioned 
advantages of virtual microscopy. In particular, it would be desirable to provide a 
method and apparatus that gives an indication of the position of an area of a 
15 microscope specimen, contained within the field of view of a microscope, within 
the complete specimen. 

[0010] One method for determining the position of an area of a microscope 
specimen contained within the field of view of a microscope within the complete 
20 specimen is to use an expensive robotic microscope stage which has a stage 
driver and sensors in order to determine the position of the field of view using the 
stage coordinates. However this technique is complex and requires expensive 
equipment not readily available to an average microscope user. 

25 [0011] The present invention therefore further aims to obviate the need for 
such expensive equipment when determining the positional information. 
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SUMMARY OF THE INVENTION 
[0012] According to a first aspect, the present invention provides a method for 
determining a position of an area of an object within said object, wherein an 
image of said area of said object is contained within a field of view of a 
5 microscope, and wherein image data representing a low magnification image of 
the complete object is available, the method comprising acquiring high 
magnification digital image data representing an image of the field of view of a 
microscope; processing the high magnification digital image data to reduce the 
resolution thereof; comparing the processed high magnification digital image data 
10 with portions of the low magnification image data, and, determining said position 
to correspond to the position of a closest matched portion of the low 
magnification image data. 

[0013] The determined position can thus be provided to the user, for example 
15 by displaying an image of the complete specimen, using the low magnification 
image data, on a display screen and highlighting the area of the image 
corresponding to the field of view. 

[0014] Advantageously, the method can be carried out without special 
20 equipment. For instance, the low magnification image data may be acquired 
using a conventional scanner; the high magnification image data may be 
acquired using a digital camera in conjunction with a conventional microscope, 
and the processing of the image data may be performed by a conventional 
computer or similar processing device. 

25 

[0015] According to a second aspect, the present invention provides a 
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computer readable medium including a computer program comprising: a program 
step for receiving high magnification image data representing the image of a part 
of an object contained in the field of view of a microscope; a program step for 
processing the high magnification image data to reduce the resolution thereof; a 
5 program step for comparing the processed high magnification image data with 
portions of low magnification image data for the complete object stored in a data 
store, and, a program step for determining the position based on the results of 
the comparison. 

10 [0016] Other preferred features and advantages of the present invention will 
be apparent from the following description and accompanying claims. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0017] Embodiments of the present invention will now be described, by way of 
15 example, with reference to the accompanying drawings, in which: 

[0018] Figure 1 illustrates an apparatus in accordance with an embodiment of 
the present invention; 

20 [0019] Figure 2 is a flow diagram illustrating the method of a preferred 
embodiment of the present invention; 

[0020] Figure 3 is a flow diagram illustrating the program steps carried out by 
a preferred computer program used to implement the method of the preferred 
25 embodiment of the present invention; 
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[0021] Figure 4 shows a low magnification image of a specimen for use in the 
method of the present invention, and 

[0022] Figure 5 is a screen shot illustrating the presentation, to a user, of the 
5 position of a high magnification image of a part of the specimen of Figure 4 within 
the complete specimen, determined in accordance with the present invention. 



DETAILED DESCRIPTION 
[0023] Figure 1 shows an example of the apparatus used to implement the 
10 present invention. Except where otherwise stated, the apparatus takes 
conventional form and is typically apparatus that is readily available to a 
microscope user. 



[0024] The apparatus comprises a microscope 1 including a movable stage 3, 
15 and a digital camera 5 connected to a camera attachment of the microscope 1 for 
capturing images of the microscope field of view. It should be noted that, as well 
as the camera attachment, the microscope 1 has a separate eyepiece or 
eyepieces (not shown) to enable a user to use the microscope 1 whilst the 
camera 5 is attached thereto. 

20 

[0025] The digital camera 5 is also coupled to provide digital microscope 
images to a workstation 7, which may be any conventional computer having a 
processor and memory, that is capable of receiving and storing the digital image 
data from the camera 5. The workstation 7 has user interfaces including a 
25 display screen 9, a keyboard 11 and a mouse 15, and peripheral devices 
including a printer 17 (not shown) and a flatbed scanner 19. 
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[0026] The workstation 7 is loaded with a computer program in accordance 
with a preferred embodiment of the present invention. In particular, the computer 
program implements a method that determines positional information relating to 
5 an object under the microscope, namely, the position of an area of an object, the 
image of which is contained within a field of view of the microscope 1 , relative to 
the complete object. In addition, the computer program provides the determined 
positional information on the display screen 9, as described in further detail 
below. 

10 

[0027] Figure 2 is a flow diagram of a preferred method for determining a 
position of an area of a specimen on a microscope slide 21 (as shown in Figure 
1), the image of which is contained within a field of view of the microscope 1 (as 
shown in Figure 1), relative to the complete specimen, in accordance with the 
15 present invention. 

[0028] At step 10, low magnification image data of the complete specimen is 
acquired and stored in memory in workstation 7. Figure 4 depicts, by way of 
example, low magnification image data, for a biological specimen on the slide 21, 
20 acquired by step 10. 

[0029] Typically step 10 is performed by scanning the microscope slide 21, 
containing the specimen, without magnification or enlargement of the image, 
using the flatbed scanner 19 to achieve the highest possible image resolution for 
25 the scanner (e.g. 600, 1200 or 2400 dpi). Since there is no magnification, the 
quantity of image data is kept to a manageable size (e.g. 4000 pixels square for 
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a specimen of 20 mm x 20 mm) for processing. It will be appreciated, however, 
that any appropriate manner of acquiring high resolution digital image data 
without significant magnification may be used, for example, scanning using a 
digital photocopier or taking a single image of the complete specimen using a 
5 high quality digital camera that can provide similar resolutions to a scanner, as 
specified above. It will be further appreciated that low level magnification (e.g. 
use of 2x objective lens) may be desirable in some circumstances and may be 
used if the quantity of data is manageable (i.e. it can be processed in a 
reasonable length of time). 

10 

[0030] At step 20, the display screen 9 displays an image 23 of the complete 
specimen (or a significant proportion thereof) on the slide 21 represented in the 
image data acquired at step 10. This image, labelled 23 in Figure 5, provides the 
user with a quick reference to the outline shape and characteristic features of the 
15 specimen. 

[0031] At step 30, the user places the slide 21 of the specimen, as shown on 
the display screen 9, under the microscope 1 , and positions the stage 3 at an 
arbitrary starting point for viewing the specimen. 

20 [0032] At step 40 the digital camera 5 takes an image of the microscope field 
of view and forwards to the workstation 7 the high magnification image data thus 
acquired, which is stored in memory and displayed on the display screen 9 as a 
field of view image 25, as shown in Figure 5. 

25 [0033] At step 50, the workstation 7 processes the high magnification data so 
as to reduce the resolution thereof. In particular, since the high magnification 
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image data represents only a small area of the complete specimen, the 
resolution thereof is higher than the resolution of the part of the low magnification 
image data that represents an area of the specimen of the same size. 
Accordingly, the high resolution image data is processed by conventional image 
5 decimation, or similar techniques, to provide processed reduced resolution image 
data representing the field of view image. In the embodiment, the reduction in 
resolution is such that the processed image data has a similar resolution to that 
of a corresponding portion of the low magnification image data of the complete 
specimen. The correct reduction in resolution is achieved by pre-calibrating the 
10 system, as discussed below. 

[0034] At step 60, the workstation 7 compares the data for the reduced 
resolution image of the current field of view with the low magnification image data 
for the complete specimen to determine if a "possible match" can be found. The 

15 comparison may be carried out, for example, using cross correlation techniques, 
as well known to a person skilled in the art. In the preferred embodiment, the 
workstation has a predetermined threshold, for each microscope objective lens, 
that defines the percentage of identical data in the low magnification and high 
magnification image data necessary in order to for there to be a "possible match". 

20 Each time a possible match is found in the low magnification data, the position 
thereof is recorded. 

[0035] Following comparison of the reduced resolution field of view image 
data with the low magnification image data of the complete specimen, if a single 
25 possible match is found, then the location of the area of the specimen in the field 
of view is determined to correspond to that of the matched area of the image of 
the complete specimen. In this case, at step 70, the display screen 9 highlights 
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the area of the low magnification image displayed that matched with the field of 
view data, as shown by a box in Figure 4. 

[0036] If more than one possible match is found, the best or closest match 
5 may be highlighted on the display (i.e. the part of the low magnification image 
having the highest percentage of identical data to the high magnification field of 
view image data). If no match is found (i.e. no part of the low magnification 
image data has a percentage of identical data exceeding the threshold), at step 
80, the display screen 9 indicates this on or adjacent to the displayed image of 
10 the specimen (e.g. by the words "Position Unknown"). 

[0037] In a preferred embodiment, at least a part of the method of the present 
invention, as illustrated in Figure 2, is implemented in the form of a software 
application which may be provided in the form of a computer program on a 

15 computer readable medium. Such a computer readable medium may be a disk 
or other data storage device, which can be loaded in a disk drive of the 
workstation 7 of the system of Figure 1. Alternatively, if the workstation 7 is 
connected to a network, the program may be held in a server carrying a website, 
which permits downloading of the program over the Internet by the workstation. 

20 Thus, the present invention may be embodied in the form of a carrier wave with 
the computer program carried thereon. 

[0038] Figure 3 illustrates the program steps carried out by a computer 
program in accordance with a preferred embodiment of the present invention. 
25 The computer program is loaded in workstation 7, which receives image data 
from digital camera 5 and flatbed scanner 19 as discussed above. 
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[0039] It is assumed that prior to the start of the program illustrated in Figure 
3, the user initiates the system, and, if necessary, performs calibration of the 
system (this is usually only required when the program is used with a system for 
5 the first time, or new equipment is installed in the system). 

[0040] Conventional calibration of each objective lens of the microscope in 
conjunction with the camera, and calibration of the scanner is necessary, and the 
skilled person would be familiar with the necessary techniques involved, and a 

10 detailed discussion thereof is unnecessary. Briefly, a calibrated graticule is used 
with the microscope and camera to measure the number of pixels per unit length 
represented in the high magnification image data, and thus the resolution thereof. 
Similarly, a calibrated graticule is used with the scanner to measure the number 
of pixels per unit length represented in the low magnification image data, and 

15 thus the resolution thereof. From this calibration information, the program 
calculates the relative resolutions of the image data for the microscope and the 
camera. 

[0041] The program is initiated by a user, and low magnification image data 
20 from the flatbed scanner 19 or other image data source is received and stored in 
the workstation memory and displayed on display screen 9 using conventional 
software (step 10 of the method of Figure 2) prior to the program steps illustrated 
in Figure 3. It will be appreciated that in other embodiments, the computer 
program may include the necessary program steps to perform the scanning step. 

25 

[0042] In accordance with a preferred embodiment, the camera is set up to 
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capture field of view images at its highest speed (e.g. 30 frames per second) and 
the program runs continuously, following initiation by the user for a particular 
specimen. Thus, the workstation continuously receives field of view images. 

5 [0043] At step 100, the program receives field of view image data from the 
digital camera 5. Any field of view image data received from camera 5 during 
subsequent processing steps 110 to 170 are not processed and may be 
overwritten in workstation memory. The field of view image data currently being 
processed is retained in memory (and displayed on the display screen 9) until the 
10 program returns to step 100, as described below. 

[0044] At step 110, the program reduces the resolution of the field of view 
image data to a resolution approximating that of a corresponding area of the 
image data of the complete specimen received from the scanner 19 and stored in 
15 memory, to obtain reduced resolution field of view image data. The reduction in 
resolution is determined based on the calibration data for the scanner and the 
objective lens of the microscope and camera from which the image data is 
acquired, as discussed above. 

20 [0045] At step 120, the program performs cross correlation of the reduced 
resolution field of view image data obtained at step 110 with the low 
magnification image data of the complete specimen stored in memory, and 
stores the location of portions of the low magnification image data for each 
possible match. As explained above, it will be appreciated that a possible match 

25 is determined based on the quantity of identical data in both sets of image data, a 
possible match being identified if the proportion of identical data exceeds a 
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preset threshold for each objective lens of the microscope. For example, a 
percentage of identical data that may be considered to be a threshold for a 
potential match may be 25%. In practice, a match is likely to exist if the 
percentage of identical data is in the range of 50% to 90% (it should be 
5 appreciated that a 100% match is statistically improbable). In the preferred 
embodiment, the user is able to adjust this threshold based on the results, and 
thus interactively adjust the system to provide the most accurate position 
determination. Thus, if the threshold is set too high, the system will not be able 
to locate the position in most cases, and the user is able to reduce the threshold 
10 in order to achieve better position determination. 

[0046] At step 130, the program considers if the cross correlation resulted in 
at least one possible match between the reduced resolution field of view data 
and the low magnification data for the complete specimen. 

15 

[0047] If step 130 determines that the cross correlation resulted in a single 
possible match of the field of view image data in the overall specimen image 
data, the program continues with step 140, and sends a signal to the display 
screen to display an indication of the location of the matched portion of the image 
20 of the complete specimen (e.g. by highlighting in colour, or, as shown in Figure 5, 
surrounding by a box, the area of the complete specimen image on the display 
corresponding to the matched image data). 

[0048] Alternatively, if step 130 determines that the cross correlation did not 
25 result in a single match of the image data, the program continues with step 150, 
by considering whether more than one possible match of the image data was 
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found by the cross correlation at step 120. 

[0049] If step 150 determines that the cross correlation performed at step 120 
did not result in more than one possible match, then no possible match has been 
5 found, and at step 170, the program sends a signal to the display screen 9 to 
display that the location of the field of view cannot be determined, in the present 
example, by displaying the words "Position Unknown". 

[0050] Alternatively, if step 150 determines that the cross correlation 
10 performed at step 120 did result in more than one match (which is unlikely for 
most biological specimens, but may occur for other types of objects), in 
accordance with the preferred embodiment, at step 160, the program selects the 
closest matched area of the low magnification image (i.e. the matched area of 
the low magnification image data with the highest percentage of identical data to 
15 the reduced resolution field of view data), and sends a signal to the display 
screen 9 to highlight this closest matched area in the complete specimen image. 
In this case, the manner of highlighting is preferably different from the 
highlighting used in response to step 140 where a single possible match is found. 

20 [0051] In another embodiment, step 160 may instead send a signal to the 
display screen 9 to highlight all the matched portions of the image of the 
complete specimen. In this case, again, an alternative form of highlighting is 
desirable to distinguish from the definite match at step 140. 

25 [0052] After the program has sent a signal to the display screen 9, at step 
140, 160 or 170, the program returns to step 100 and commences processing the 
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next field of view image to be received from the camera 5 which is then 
immediately displayed on the display screen 9 and stored in memory. 

[0053] In view of the likelihood of only a single match occurring for certain 
5 types of object, in another embodiment, steps 150 and 160 may be omitted. 
Thus, in this embodiment, if step 130 determines that the cross correlation did 
result in a (single) match, the program proceeds to step 140, and if step 130 
determines that the cross correlation did not result in a (single) match, the 
program proceeds straight to step 170. 

10 

[0054] As mentioned above, Figure 4 illustrates, by way of example, the 
image of a specimen on slide 21 acquired by the low magnification scan in 
accordance with the present invention. This low magnification image, or a part 
thereof, is displayed on display screen 9, as shown in Figure 5 and described 
15 below, when the program of Figure 3 is used in conjunction with the system 
shown in Figure 1 . 

[0055] As shown in Figure 5, in another embodiment, the display screen 9 
includes two separate display areas; a main area displays the current, magnified, 
20 high resolution field of view image of the microscope 1 , labelled 25 in Figure 5, 
and a secondary area displays the low magnification image of the complete 
specimen (or a proportion thereof), labelled 23 in Figure 5. The display may also 
include a toolbar 17 and an information area 29 providing identifying information 
about the current specimen and/or system equipment. 

25 

[0056] In the preferred embodiment, and as shown in Figure 5, when the 
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program is in use, the displayed low magnification image 23 does not show the 
complete specimen (as shown in Figure 4) but rather shows a significant 
proportion of the area of the specimen surrounding the identified position of the 
current field of view image 25. It will be appreciated that initially, or when using a 
5 lower magnification objective lens (in the illustrated example, a 10x objective lens 
is used to obtain the field of view image), the complete specimen may be shown 
in the secondary area of the display, and used as the navigation map. 

[0057] As shown in Figure 5, an area is highlighted by dashed outline in the 
10 low magnification image of the specimen 23, which area corresponds to, and 
represents the position within the complete specimen of, the high magnification 
field of view image 25. The highlighted area changes each time the position of 
the stage 3 of the microscope 1, and thus the specimen, is moved. In this way, 
displayed image 23 provides a navigation map of the specimen which enables 
15 the user to easily locate the current position of the field of view image, and to 
determine the relative positions of other features of interest in the specimen. 

[0058] Whilst one embodiment of the present invention has been described 
for use with an optical microscope, it will be appreciated that the present 
20 invention may be used with other types of microscope. 

[0059] Various modifications and changes may be made to the described 
embodiments. It is intended to include all such variations, modifications and 
equivalents that fall within the spirit and scope of the present invention. 

25 
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